Mixed reality (MR) or augmented interactive input systems are known in the art. Some existing augmented interactive input systems make use of a document camera to track and image physical objects such as cubes and flashcards that comprise recognizable patterns. Digital content is superimposed on top of the images captured by the document camera and the digital content and images are presented on a display. The direct association of the physical objects to digital content allows users to manipulate the digital content through manipulation of the physical objects. Unfortunately, these augmented interactive input systems have only permitted the position and orientation of physical objects to be tracked. As a result, user manipulations of digital content in this scenario have been limited to translations and rotations of physical objects. By only permitting the position and orientation of the physical objects to be tracked, interactions such as selecting or clicking a button or opening a menu have been difficult to realize without use of an external device such as a mouse or keyboard.
Another problem that occurs in existing augmented interactive input systems is the temporary loss of physical objects that are being tracked. When an augmented interactive input system loses track of a physical object and then regains tracking of the physical object, the associated digital content shown on the display disappears and then quickly re-appears, causing unwanted flickering.
The publication entitled “Occlusion based Interaction Methods for Tangible Augmented Reality Environments,” authored by Lee et al. and published in the Proceedings of the 2004 ACM SIGGRAPH international conference on Virtual Reality continuum and its applications in industry (pages 419-426, 2004), discloses an occlusion based interaction system and method for interacting with two-dimensional (2D) digital objects in a camera-based tangible augmented reality environment. A marker comprising a set of patterns is imaged by a camera, and patterns on the marker are recognized. When a user uses a finger to manipulate the marker and the finger occludes at least one pattern on the marker, the interaction system determines which pattern has been occluded, and then performs a 2D action (e.g., clicking a button or opening a menu) accordingly. Because multiple patterns are required, either the marker has to be sufficiently large so as to accommodate the patterns, making the marker, as a physical object, difficult to manipulate, or the patterns have to be small, which may increase the failure rate of pattern recognition.
The publication entitled “The Bubble Cursor: Enhancing Target Acquisition by Dynamic Resizing of the Cursor's Activation Area,” by Grossman, et al. and published in CHI 2005 Conference Proceedings: ACM Conference on Human Factors in Computing Systems, pages 281-290, discloses a bubble cursor used on a 2D graphic user interface (GUI). The bubble cursor has a circle shape, and captures the target whose border is closest to the cursor's center.
Although mixed reality or augmented interactive input systems have been considered, improvements in mixed reality or augmented interactive input systems are desired. It is therefore an object of the following to provide a novel interactive input system having a three-dimensional input space, and novel visual compensation methods, 3D tools and gestures for interacting in mixed reality environments.